Skip to content

feat(core): add Bluestreak protocol#129

Merged
polinikita merged 18 commits intomainfrom
feat/bluestreak
Mar 20, 2026
Merged

feat(core): add Bluestreak protocol#129
polinikita merged 18 commits intomainfrom
feat/bluestreak

Conversation

@polinikita
Copy link
Member

@polinikita polinikita commented Mar 19, 2026

Summary

  • add the new Bluestreak consensus protocol and rename the original MysticetiCompress implementation to its final public name
  • extend block headers and digests with unprovable_certificate, plus Bluestreak-specific block validation rules
  • compress non-leader references, propagate clean-vertex certification through the dual DAG, and use that path for recovery, reachability, and parent gating
  • add Bluestreak direct commit and skip logic, including traversal over unprovable_certificate edges during commit and linearization
  • wire Bluestreak through networking, validator smoke coverage, orchestrator and dryrun entry points, and default its dissemination mode to pull

Test plan

  • cargo check --all-features
  • cargo test -p starfish-core --lib bluestreak

polinikita and others added 9 commits March 19, 2026 17:45
Add MysticetiCompress to ConsensusProtocol enum with from_str parsing,
helper methods (is_mysticeti_compress, uses_dual_dag), and wire into
all match arms: PushCausal dissemination, no erasure coding, Blake3
transaction commitment, ancestor-based subdag collection, full-block
format, wave_length=3 with pipeline=true (3 pipelined committers).
Add Optional<BlockReference> field for MysticetiCompress unprovable
certificate mechanism. Includes accessors on BlockHeader and
VerifiedBlock. All existing callers pass None. Digest integration
deferred to block building step.
…cetiCompress

Add optional unprovable_certificate field to BlockHeader to support
the MysticetiCompress protocol. This field references a leader block at
round r-2 that is claimed to have been seen by 2f+1 validators at
round r-1, and is part of the signed block hash for compression.

Changes:
- Add unprovable_certificate: Option<BlockReference> field to BlockHeader
- Add accessor methods unprovable_certificate() to BlockHeader and VerifiedBlock
- Initialize field to None in all BlockHeader constructors
- Update BlockDigest hashing to include unprovable_certificate in digest
- Update all Block Digest::new and new_without_transactions calls to pass parameter
- Update Signer::sign_block and PublicKey::verify_signature_in_block to handle field

Co-Authored-By: Claude <noreply@anthropic.com>
MysticetiCompress non-leaders include at most own_prev + leader at r-1.
Leaders keep the full clean-DAG frontier. compute_unprovable_certificate
checks direct evidence (2f+1 votes at r-1) and propagation (f+1 at r).
Replace SailfishPlusPlus-specific checks with uses_dual_dag() helper:
- Certified ref recovery now applies to both SailfishPlusPlus and MysticetiCompress
- proposal_round logic for vertex certification applies to both dual DAG protocols

This enables MysticetiCompress to benefit from the same vertex certification
and proposal timing mechanisms as SailfishPlusPlus.

Co-Authored-By: Claude <noreply@anthropic.com>
Replace explicit SailfishPlusPlus checks with uses_dual_dag() helper
in dag_state.rs to support both SailfishPlusPlus and MysticetiCompress
protocols for:
- Vertex certification recovery from storage
- Proposal round calculation (requires certified vertices)
- Certified parent quorum checks

This enables MysticetiCompress to benefit from the dual DAG
infrastructure (clean/dirty DAG with vertex certification).

Co-Authored-By: Claude <noreply@anthropic.com>
…etiCompress

Ensure unprovable_certificate targets are included in missing
dependencies check. When a MysticetiCompress block references an
unprovable certificate, that certificate block must be present
in the vertex certification set before the block can be processed.

This maintains ancestor-closure invariant for the clean DAG.

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions
Copy link

github-actions bot commented Mar 19, 2026

Code Coverage

Package Line Rate Health
crates/starfish-core/src/core_thread 0%
crates/starfish-core/src/runtime 0%
crates/orchestrator/src 16%
crates/starfish-core/src/consensus 24%
crates/starfish/src 3%
crates/orchestrator/src/protocol 1%
crates/starfish-core/src 33%
crates/orchestrator/src/client 7%
Summary 29% (3478 / 12116)

- Add unprovable_certificate edge traversal to
  infer_vertex_certificate_closure, matching the linearizer pattern.
  Without this, inferred certification could activate a block whose
  unprovable_certificate target was not yet certified.
- Remove dead duplicate Bluestreak branch in
  compress_pending_block_references (unreachable after the early-return
  block added at the top of the function).
- Remove timed!() wrappers from try_new_block internals (sub-function
  timers removed from measurements fixture accordingly).
- Update grafana dashboard stacking mode for utilization panel.
Remove outer wrapping timers (Core::add_blocks, Core::add_headers,
Syncer::try_new_commit, Core::try_new_commit, Committer::direct_decide,
Committer::indirect_decide) that double-counted time with their inner
leaf timers. Add new leaf timers for Core::try_commit, Core::cleanup,
and Core::handle_committed_subdag. The remaining timer set partitions
core thread time so their sum approximates core_lock_util.

Update Grafana dashboard query to match only active timer prefixes.
Replace linear .iter().any() scans in proposal_round() and
certified_parent_quorum() with O(log n) BTreeSet::range() queries,
exploiting BlockReference's (round, authority, digest) ordering.
Read back from self.decided at the top of each round iteration,
matching the Mysticeti path. Skip and Commit are monotonically
final, so cache hits avoid re-evaluating skip evidence and
re-walking the commit chain.
Bluestreak leaders are the sole aggregation points in the DAG, but
compress_pending_block_references was filtering out refs from rounds
r-2 or earlier. This caused late-arriving blocks to be silently dropped
and become permanently unreachable. Remove the r.round >= prev_round
filter so leaders include all non-self pending refs, consistent with
StarfishBls leader behavior.
Cached Skips should short-circuit evaluation but not be pushed to the
committed list, matching the original guard behavior. Restructure the
decided-cache fast path to only push LeaderStatus::Commit entries.
…onfig panel

- Core functions time/s panel now shows avg by proc across all validators
  with stacked area chart
- Configuration panel target order matches title: protocol, dissemination,
  tx type, storage
@polinikita polinikita merged commit be4441a into main Mar 20, 2026
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant